Fixes #16018: Existential widening for wildcard arguments#26152
Open
He-Pin wants to merge 2 commits into
Open
Conversation
Contributor
Author
|
This fix will facilitate cross-compilation between Scala 2.13 and Scala 3, especially for Java users. |
He-Pin
commented
May 23, 2026
He-Pin
commented
May 23, 2026
The Scala 3 typer was rejecting subtypings of the form
F[? >: lo <: hi] <: F[X]
for covariant or contravariant `F`, even when `X = hi` (covariant) or
`X = lo` (contravariant). This is a regression against Scala 2 and surfaced
as the akka-derived report in scala#16018:
def f[M](xs: java.lang.Iterable[? <: Container[Any, ? <: M]])
: Seq[Container[Any, M]] =
seqOf(xs).collect {
case g: SubContainer[Any, M] @unchecked => g
case other => other
}
Once `Inferencing.captureWildcards` lifted the Java wildcard to a
`TypeBox.CAP` skolem and pattern matching narrowed the first case body
to `pat ∩ ?N.CAP`, the inferred result reduced to a chain of subtype
checks that ultimately required `F[? <: M] <:< F[M]` for covariant `F`.
That check went through `TypeComparer.compareCaptured` and was answered
in the negative.
Type-theoretic justification
----------------------------
For a covariant type constructor `F` and an existential `∃X. X <: hi`,
⨆{F[X] | X <: hi} = F[hi] (covariant supremum)
— the supremum is attained at `X = hi`, by covariance. The compiler must
admit that supremum as a subtype of `F[hi]`. Dually for contravariance with
the lower bound:
⨅{G[X] | X >: lo} = G[lo] (contravariant infimum)
This is the standard existential-elimination rule for variant occurrences.
Bug
---
`compareCaptured` was checking
v > 0: isSubType(paramBounds(tparam).hi, arg2)
v < 0: isSubType(arg2, paramBounds(tparam).lo)
i.e. it asked whether the *declared parameter bound* (which collapses to
`Any` / `Nothing` for unconstrained type parameters) conforms to `arg2`,
instead of asking whether the *wildcard's own bound* (`arg1.hi` / `arg1.lo`)
does. That rejected every interesting case.
Fix
---
In `TypeComparer.compareCaptured` use the wildcard's own hi/lo, intersected
(resp. unioned) with the declared parameter bounds to preserve soundness in
the corner case where a wildcard is wider than its parameter permits:
v > 0: isSubType(arg1.hi & paramBounds(tparam).hi, arg2)
v < 0: isSubType(arg2, arg1.lo | paramBounds(tparam).lo)
Tests (directional matrix)
--------------------------
- `tests/pos/i16018.scala` — mbovel's minimization plus a pure-Scala matrix
that exercises subtype patterns + fallback `case other` over wildcards.
- `tests/pos/i16018b.scala` — the akka-style shapes that motivated the
ticket: `java.lang.Iterable[? <: G[? <: M]]` / `java.util.List[…]`
routed through `seqOf`, then `collect` / `map` with subtype + fallback
patterns.
- `tests/pos/i16018c.scala` — positive half of the variance × bound matrix
for the fix itself: covariant + upper, contravariant + lower, nested,
via method type params, with parameter-bound tightening.
- `tests/neg/i16018c.scala` — negative half of the matrix: invariant
containers, covariant + lower, contravariant + upper, and the soundness
guard against wildcards wider than the required type. These must remain
rejected after the fix.
Verified locally
----------------
- All four `i16018*` tests: 16/16 pass via
`sbt 'scala3-compiler-bootstrapped/testOnly … CompilationTests'`
with `-Ddotty.tests.filter=i16018`.
- `wildcard` / `match` / `variance` filter subsets: 16/16 each.
- Full `CompilationTests` run: the remaining 4 failures
(`tests/run/i13358.scala`, `tests/run/lazy-*.scala`,
`tests/run/t5552.scala`, `tests/run/t7406.scala`,
`tests/run/isInstanceOf-eval.scala`,
`tests/run/i24553.scala`,
`tests/pos-custom-args/captures/fill-cbn.scala`, and the
capture-checking neg suite) reproduce on `main` without this change and
are caused by JDK 25 environment drift (`sun.misc.Unsafe` deprecation
messages on stdout, the introduction of `java.lang.IO`, and removal of
`native` from `Object.wait`).
…ounds Motivation ---------- The widening fix in b952dfd admits subtypings for F-bounded type constructors such as `class C[+T <: C[T]]` at the surface, but the resulting typed tree fails `-Ycheck:all` under frozen constraints with assertions of the form `M <:< xs.T` / `C[M] <:< C[xs.T]`. Without a guard, this surface-pass / Ycheck-fail combination silently produces an internally inconsistent typed tree (compile reports 0 errors). Modification ------------ In `TypeComparer.isSubArgs`, precompute `recursiveParamBounds`: the set of `tparams2` whose declared bounds recursively refer to themselves. The detection visits `tparam.info` via `TypeAccumulator`: - direct `TypeRef` to the parameter -> recursive - any `LazyRef` -> recursive (conservative, avoids forcing recursive class-header init) `compareCaptured` consults `hasRecursiveParamBounds(tparam)` *before* the stable capture conversion branch, so both stable capture and existential widening are short-circuited for recursive bounds, falling back to the pre-fix conservative `false`. The set is cached via `lazy val` to amortise across arg positions in the same `isSubArgs` invocation. Tests ----- - `tests/neg/i16018d.scala` -- F-bound guard single-point pin (`class FBounded[+T <: FBounded[T]]`). Guard necessity was verified out-of-band via a temporary `if (false && hasRecursiveParamBounds...)`: with the guard disabled, this test compiles with 0 errors. - `tests/pos/i16018d.scala` -- four cases not covered by the original directional matrix: Function1 nested under Co, intersection upper bound, path-dependent upper bound, and acyclic dependent bound (reverse-pin proving the guard does not over-restrict to all dependent bounds, only ones with a self-reference). - `tests/neg/i16018e.scala` -- three HKT-nested variance cases (Function1 contra+upper, Function1 co+lower, Inv-in-Co) pinning per-position variance checks through HKT nesting. Verified locally ---------------- `testOnly dotty.tools.dotc.CompilationTests -- -Ddotty.tests.filter=i16018` -> 16 passed, 0 failed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #16018
A long-standing wildcard-subtyping gap in
TypeComparer.compareCapturedwas rejectingF[? <: hi] <:< F[hi]for covariantF(dually for contravariantG), even though existential elimination over a variant type constructor admits exactly that widening. The fix consults the wildcard's own bound, intersected with the parameter's declared bound, instead of the declared bound alone — purely additive, mirrors the same lattice operation Scala 2 implements, and closes the original akka-derived ticket along with the broader pattern.How much have you relied on LLM-based tools in this contribution?
Extensively, for: minimizing the akka reproducer, instrumenting
TypeComparerto localize the failing subtype path, identifying the wrong-field consultation (paramBounds(tparam).hivsarg1.hi), proving the fix is type-theoretically correct, and designing the directional pos / neg matrix. I have personally reviewed every line of the diff, can explain each step, and have stress-tested the fix against the full localCompilationTestssuite plus targetedwildcard/match/variancefilter subsets before submitting.How was the solution tested?
New automated tests (including the issue's reproducer).
Run via
sbt 'scala3-compiler-bootstrapped/testOnly dotty.tools.dotc.CompilationTests':i16018filter — all five test files (16/16 pass).wildcard/match/variancefilter subsets — 16/16 each.CompilationTestsrun — the residual failures (tests/run/i13358.scala,tests/run/lazy-*.scala,tests/run/t5552.scala,tests/run/t7406.scala,tests/run/isInstanceOf-eval.scala,tests/run/i24553.scala,tests/pos-custom-args/captures/fill-cbn.scala, capture-checking neg suite) all reproduce onmainwithout this change and are caused by JDK 25 environment drift (sun.misc.Unsafedeprecation messages on stdout, the introduction ofjava.lang.IO, removal ofnativefromObject.wait). Verified by re-running them on a cleanmaincheckout.Diagnosis — how the bug was located
The ticket and its successive minimizations all reduced to the same symptom: a
collect { case g: Sub[…] @unchecked => g; case other => other }over a value derived from ajava.lang.Iterable[? <: G[? <: M]]rejects the second case withInstrumentation of
TypeComparer.compareTypeParamRefandaddOneBoundexposed two layered issues:Layer 1 — sequential constraint accumulation on the partial-function result variable
B. The first case body's pattern-narrowed typeSub[…] ∩ ?N.CAPwas added asB's lower bound, after which the second case body's check?N.CAP <:< Bwas rejected because?N.CAP ⊄ Sub[…] ∩ ?N.CAP. The fix point we actually want isB := ⨆ᵢ T_i(the LUB of case body types), namely?N.CAP.Layer 2 — and this turned out to be the root cause — even bypassing layer 1, the surrounding adaptation reduced to
F[? <: M] <:< F[M]for covariantF(after Seq's covariance). That check went throughTypeComparer.compareCapturedand was rejected. A reduced standalone reproduction confirmed the regression independently of pattern matching:So the partial-function symptom was downstream of a more fundamental gap in wildcard subtyping under variance. Fixing layer 2 dissolves layer 1, and is what this PR does.
Root cause
TypeComparer.compareCapturedis the fallback for an applied-type argument check when the source argument is a wildcardarg1: TypeBoundsand the target argument is a concretearg2. Pre-fix it asked— i.e. it consulted the declared parameter bound, which for an unconstrained
Tcollapses toAny(covariant case) orNothing(contravariant case), so the conformance check becameAny <: arg2and almost always failed. The wildcard's own bound (arg1.hi/arg1.lo) — which is the only place where the user's intent? <: hiwas recorded — was never consulted.Type-theoretic justification
Order subtypes (
<:) so that larger means more general. For each variance direction, the existentialF[? >: lo <: hi]denotes a family of types parameterised by the witnessX, and we ask for the least upper bound (⨆) of that family — the smallest type into which every member of the family can be widened. The fact that a single concrete type is the LUB is what makes the widening rule sound (admissibility of existential elimination at a variant position).Covariant
F[+T]. The family is{F[X] | X <: hi}. By covariance,so the family is order-preserving in
X; its LUB is attained atX = hi:Contravariant
G[-T]. The family is{G[X] | X >: lo}. By contravariance,so the family is order-reversing in
X; its LUB is attained atX = lo(note: the supremum sits at the small end of the witness, because the type constructor flips the order):Both rules use the same lattice operation (LUB) and the same notion of "widening"; the only thing that differs is which witness attains the supremum, because the variance of
F/Gdecides whether the witness ordering and the result ordering agree or disagree.Invariant positions admit neither widening (the family
{F[X] | X <: hi}for invariantFhas no nontrivial supremum in the family itself), and mixed variance / bound combinations (covariant + lower, contravariant + upper) similarly have no usable LUB — those cases must remain rejected. This is whattests/neg/i16018c.scalapins.Scala 2 implements both rules; Scala 3 had regressed.
Fix
In
TypeComparer.compareCaptured, consult the wildcard's own bound, intersected (resp. unioned) with the parameter's declared bound. The intersection is the declared-bound guard (a completeness guard, not a soundness one — without itNAIVE = arg1.hiwould produce false negatives like rejectingCoBounded[? <: Any] <:< CoBounded[Holder]): if a wildcard is wider than what the parameter permits (e.g.F[T <: Holder]applied to? <: Any), the effective bound is the tighter of the two.This is purely additive: it never accepts a subtyping that was already accepted by a different path; it only admits the cases the LUB rule above justifies. Three implementations are distinguishable on the test matrix below:
OLDignores the wildcard's bound: for unboundedCo[+T]fed? <: Mit asksAny <: Mand rejects a valid widening.NAIVEignores the parameter's declared bound: forCoBounded[+T <: Holder]fed? <: Anyit asksAny <: Holderand rejects another valid widening. All three are sound (none accepts a subtyping that violates the bounds either party records); the distinction is completeness.THE FIXis the intersection — the only one of the three that is also complete on this fragment, admitting every existential elimination justified by variance + bounds.Tests (directional matrix)
tests/pos/i16018.scalacase othertests/pos/i16018b.scalajava.lang.Iterable[? <: G[? <: M]]/java.util.List[…]routed throughseqOf, thencollect/maptests/pos/i16018-orig.scala_→?syntax migration), so the original ticket cannot regresstests/pos/i16018c.scalaOLDfromTHE FIX(the covariant headline and its dual); §4–5 distinguish a hypotheticalNAIVErefactor (drop the intersection / union) fromTHE FIX— they pin the declared-bound guard (NAIVE =arg1.hialone produces false negatives like rejectingCoBounded[? <: Any] <:< CoBounded[Holder]; THE FIX accepts it) so future code archaeology cannot quietly delete the& paramBounds/| paramBoundstests/neg/i16018c.scalaCoBounded[? <: Any]must conform atCoBounded[Holder](pinned positively) but not atCoBounded[Sub]forSub <: Holder(a strict subtype is out of range, not unsound); dual for the contravariant lo case. These boundary checks pin the upper / lower stop of the widening (all three implementations reject them), not the distinguishing behaviour between OLD / NAIVE / THE FIXAny future refactor of
compareCapturedwill turn one of the two halves red the moment any of the three implementations above stops behaving as required.